<html>
<head><meta charset="utf-8"><title>LLVM upgrade effort · t-compiler/wg-llvm · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/index.html">t-compiler/wg-llvm</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html">LLVM upgrade effort</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="181073103"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181073103" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> J. Ryan Stinnett <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181073103">(Nov 19 2019 at 01:07)</a>:</h4>
<p>I am trying to get a sense of how much pain / time / effort is required each time Rust goes through an upgrade (not just a cherry picking a bug fix) of LLVM. Along with that, I am curious about the kinds of changes typically required to adapt to the new version of LLVM.</p>
<p>Looking over the docs in the <a href="https://rust-lang.github.io/rustc-guide/codegen/updating-llvm.html" target="_blank" title="https://rust-lang.github.io/rustc-guide/codegen/updating-llvm.html">rustc guide</a>, it feels like quite an involved process that someone might be focused on full time for a while and seems to occur a few times per year for specific features and also to stay current with each LLVM release. Is most of the time spent adapting the Rust &lt;-&gt; LLVM interface for API changes, or is it more dominated by fighting surprise behavior changes in LLVM and finding workarounds?</p>
<p>I have several ideas for tooling to help with these LLVM upgrade tasks, but I want to check whether my assumptions match the experience of those who've worked through an upgrade in past for Rust.</p>



<a name="181075972"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181075972" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181075972">(Nov 19 2019 at 02:13)</a>:</h4>
<p>I think we've not historically experienced much pain from LLVM upgrades, beyond breakage -- but that's generally on tier 2 or tier 3 platforms -- <span class="user-mention" data-user-id="138448">@cuviper</span> and <span class="user-mention" data-user-id="116015">@Alex Crichton</span> may be able to say more, I think they've been involved with the last couple upgrades</p>



<a name="181076344"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181076344" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181076344">(Nov 19 2019 at 02:22)</a>:</h4>
<p>From what I remember 99% of the pain of an upgrade is fixing regressions in the ci builds we see through bors</p>



<a name="181076347"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181076347" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181076347">(Nov 19 2019 at 02:22)</a>:</h4>
<p>The API changes are mostly quite smooth</p>



<a name="181076349"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181076349" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181076349">(Nov 19 2019 at 02:22)</a>:</h4>
<p>Sometimes it's taken forever to narrow down issues</p>



<a name="181076350"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181076350" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181076350">(Nov 19 2019 at 02:22)</a>:</h4>
<p>Other times it's been super smooth</p>



<a name="181076357"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181076357" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181076357">(Nov 19 2019 at 02:22)</a>:</h4>
<p>I haven't been too too involved in the last one or two upgrades though, and I think they were relatively painless</p>



<a name="181090437"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181090437" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> centril <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181090437">(Nov 19 2019 at 08:17)</a>:</h4>
<p>(Don't know how relevant it is, but generally speaking, having frequent LLVM submodule bumps is undesirable because all the rust-lang/rust devs have to rebuild LLVM, which is painful)</p>



<a name="181104499"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181104499" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> J. Ryan Stinnett <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181104499">(Nov 19 2019 at 11:57)</a>:</h4>
<p>It sounds like it's actually easier and less painful than I assumed then. <span aria-label="happy" class="emoji emoji-1f600" role="img" title="happy">:happy:</span> (The text "Unlike bugfixes, updating to pick up a new feature of LLVM typically requires a lot more work." in the rustc guide led me to assume it was usually a long / arduous adventure.)</p>
<p>Good to know about the additional complexity of LLVM changes forcing rebuilds for rustc devs... I hadn't realised that particular impact of more frequent LLVM upgrades.</p>



<a name="181107278"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181107278" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181107278">(Nov 19 2019 at 12:44)</a>:</h4>
<p>It's not really an impact that I think is too important, or at least I barely notice it. I do have decent hardware though :)</p>
<p>It's also something that we can (and should) fix, it's just that it requires some design work and the exact solution isn't completely clear.</p>



<a name="181108998"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181108998" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mati865 <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181108998">(Nov 19 2019 at 13:08)</a>:</h4>
<blockquote>
<p>I think we've not historically experienced much pain from LLVM upgrades, beyond breakage -- but that's generally on tier 2 or tier 3 platforms</p>
</blockquote>
<p>FYI LLVM 9 upgrade broke tier1 targets when using <em>non</em> default options: <a href="https://github.com/rust-lang/rust/issues/64609" target="_blank" title="https://github.com/rust-lang/rust/issues/64609">#64609</a> <a href="https://github.com/rust-lang/rust/issues/65618" target="_blank" title="https://github.com/rust-lang/rust/issues/65618">#65618</a></p>



<a name="181111301"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181111301" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> J. Ryan Stinnett <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181111301">(Nov 19 2019 at 13:39)</a>:</h4>
<p>For hunting down LLVM-related regressions, does that typically mean compiling many random revisions of LLVM locally to try to bisect the issue? (If so, it sounds slow since I imagine it takes a while to build at each bisection step.)</p>



<a name="181114111"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181114111" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181114111">(Nov 19 2019 at 14:14)</a>:</h4>
<p>For me I typically extract ir first to get a standalone test I can reproduce with just llvm tools, then I reduce (sometimes also with llvm tools)</p>



<a name="181114128"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181114128" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181114128">(Nov 19 2019 at 14:14)</a>:</h4>
<p>And then yeah bisection is typically next</p>



<a name="181115003"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181115003" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> J. Ryan Stinnett <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181115003">(Nov 19 2019 at 14:23)</a>:</h4>
<p>Ah right, makes sense to reproduce in LLVM standalone where possible to simplify. LLVM has also added their own <a href="https://github.com/llvm/llvm-project/tree/master/llvm/tools/llvm-reduce" target="_blank" title="https://github.com/llvm/llvm-project/tree/master/llvm/tools/llvm-reduce"><code>llvm-reduce</code> tool for IR reduction</a> recently, maybe that will be of some help here.</p>



<a name="181115713"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181115713" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> J. Ryan Stinnett <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181115713">(Nov 19 2019 at 14:30)</a>:</h4>
<p>For LLVM standalone, there's also <a href="https://github.com/llvm-mirror/zorg/blob/master/llvmbisect/docs/llvmlab_bisect.rst" target="_blank" title="https://github.com/llvm-mirror/zorg/blob/master/llvmbisect/docs/llvmlab_bisect.rst"><code>llvmlab bisect</code></a> for accelerated bisection via a shared cache.</p>



<a name="181116757"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181116757" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181116757">(Nov 19 2019 at 14:40)</a>:</h4>
<blockquote>
<p>It's not really an impact that I think is too important, or at least I barely notice it. I do have decent hardware though :)</p>
<p>It's also something that we can (and should) fix, it's just that it requires some design work and the exact solution isn't completely clear.</p>
</blockquote>
<p>LLVM builds take 30-60min on my hardware, which is not totally shabby either. Multiply this by 3 for 3 rustc checkouts (so that I can work on more than one PR at a time). It's a big pain.</p>



<a name="181116854"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181116854" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181116854">(Nov 19 2019 at 14:41)</a>:</h4>
<p>in comparison, building one stage (so that I can run stage 1 tests) <em>without</em> having to build LLVM usually takes &lt;20min.</p>



<a name="181134776"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187780-t-compiler/wg-llvm/topic/LLVM%20upgrade%20effort/near/181134776" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187780-t-compiler/wg-llvm/topic/LLVM.20upgrade.20effort.html#181134776">(Nov 19 2019 at 17:30)</a>:</h4>
<p>in theory e.g. ccache (<em>not</em> sccache) would solve the multiply by 3</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>